Week 14 Covariate Effect (Forest) Plot

Application: Simulation with Parameter Uncertainty

Shen Cheng

2024-12-06

Outline

  • Forest plot introduction
  • Hands-on

Forest plot introduction

  • Traditionally, forest plot used to display results of multiple clinical studies1.
    • Extensively used in Cross-study meta-analyses.
    • Point estimates with associated intervals are displayed.
  • Recently, its applications in pharmacometrics were documented in the FDA popPK guidance2.
    • Simulation with uncertainty on fixed effect parameters.
    • Visualize the covariate effect on simulated parameters of interest (e.g., AUC, Cmax, T>MIC, etc).

A forest plot example1

Interpretation of a forest plot1

Simulation workflow

  • Acquire parameter uncertainty distributions (covariance matrix, bootstrap, SIR or Bayesian posterior).
  • Simulate with parameter uncertainty on fixed-effect parameters for:
    • a reference subject.
    • a few non-reference subjects by changing one covariate at a time (ceteris paribus).
  • For each subject at each simulation replicate, PK/PD parameters of interest (e.g., AUC, Cmax, etc) were calculated.
  • Standardize the PK/PD parameters of each non-reference subject relative to the reference subject.
  • Plotting the simulation.

Simulation workflow1

Advantages and disadvantages1 2

  • Advantages:
    • Visually appealing and intuitively understandable.
    • Allow the assessment of covariate effect one at a time.
    • Provide uncertainty measurements around the point estimates.
      • Potential to make inferences on statistical significance and clinical relevance.
  • Disadvantages:
    • Do not account for correlation among covariates.
    • Non-plausible scenrios can be obtained by varying covariates at a time.
    • Only assessing “marginal effects”.

Forest plot with both uncertainty and variability1

Simulation workflow

  • Simulate a large and realistic covariate distribution (virtual population).
  • Pharmacometric simulations using the covariate distribution, between-subject variability and uncertainty.
  • For each subject at each simulation replicate, PK/PD parameters of interest (e.g., AUC, Cmax, etc) were calculated.
  • Stratify covariates based on quantiles, and standardize the PK/PD parameters of each non-reference subject relative to the reference subject.
  • Plotting the simulation.

Forest plot with both uncertainty and variability1

Controversy

  • The application of a forest plot constructed with both uncertainty and variability is not well-documented in the recent FDA popPK guidance.1

  • “Simulations based on uncertainty of fixed-effect parameters, BSV, and uncertainty on BSV is considered more robust and realistic, as it provides the joint effects of BSV and multiple covariates based on a database of real patients or in a virtual population with correlated covariates.”2

  • “Although it is technically feasible to use forest plots for visualizing between-subject variability, we strongly advise against it…blurring the use of the error bars leads to a significant risk of confusion for the viewer”3

R packages